di intern extent formula compact fun algorithm dynam dfast layer label propag multiresolut coordin dfree order compress social network hyperanf approxim neighbourhood function larg graph budget intern extent formula compact proceed intern confer web search web data mine effici algorithm larg dscale local triangl count articl studi problem approxim local triangl count larg graph larg graph estim accur number triangl incid node graph question undirect direct graph problem comput global number triangl graph consid knowledg contribut address problem approxim local triangl count focu effici issu aris massiv graph consid direct case distribut local number triangl relat local cluster coeffici interest applic show measur comput detect presenc spam activ largescal web graph provid featur content qualiti assess social network comput local number triangl undirect direct propos approxim algorithm base idea min wise independ permut broder al algorithm oper semi stream fashion space main memori perform log sequenti scan edg graph algorithm proceed intern confer web search web data mine theori practis monoton minim perfect hash minim perfect hash function shown compress data data manag task order preserv minim perfect hash function retriev posit kei list kei abil preserv order lead unavoid log lower bound number bit requir store function recent observ frequent kei hash sort intrins lexicograph order typic case dictionari search engin list url web graph refer restrict version problem monoton minim perfect hash analys experiment data structur propos propos method albeit asymptot equival wors perform practis provid balanc access speed eas construct space usag vote social network vote system set rule commun adopt collect decis paper studi vote system kind commun electron mediat social network focu deleg democraci proxi vote recent receiv increas interest abil combin benefit direct repres system perfectli suit electron mediat social network context vote system user express prefer peopl explicitli connect prefer propag transit attenu factor present system studi properti consider problem miss vote relev onlin network recent case show experi real world network provid interest insight signific stabil result obtain suggest vote system monoton minim perfect hash search sort tabl access minim perfect hash function map set kei set biject classic result state minim perfect hash constant time structur occupi space close lower bound log bit element problem monoton minim perfect hash biject requir preserv lexicograph order kei monoton minim perfect hash function weak form index rank set answer randomli goal minimis descript size hash function show set element univers element log log bit suffici hash monoton evalu time log altern space log bit queri time data structur improv straightforward construct log space log queri time consequ search sort tabl access tabl addit log log bit result base structur independ interest repres trie compact admit error applic structur show comput predecessor sort order arbitrari element access expect index log bit improv trivial result nw bit impli effici index search block memori permut web graph investig web graph compress clear order node graph fundament influenc compress rate express number bit link author link databas instanc investig approach extrins order url order intrins coordin free order base row adjac matrix lexicograph grai code conclud url order advantag spite small penalti compress paper approach issu systemat test order propos experi made webgraph framework show compress techniqu structur graph produc significantli result show transpos web graph url order significantli effect order combin host inform grai lexicograph order outperform previou method larg transpos graph yield incred compress rate bit link dango japanes cake queri reformul model pattern abstract understand queri reformul pattern kei step gener web search engin improv user web search experi predict intent help locat inform effect step direct build accur model classifi user queri reformul broad class gener special error correct parallel move achiev accuraci appli model automat label larg queri log creat annot queryflow graph studi result reformul pattern find result consist previou studi smaller manual annot dataset discov interest pattern includ connect reformul type topic categori final appli find queri log publicli research purpos demonstr reformul classifi lead improv recommend queri recommend system pagerank function depend pagerank defin stationari state markov chain chain obtain perturb transit matrix induc web graph damp factor spread uniformli part rank choic emin empir case origin suggest brin page paper give mathemat analysi pagerank show contrarili popular belief real world graph valu close give meaning rank give close form formula pagerank deriv order prove iter power method obtain truncat pagerank power seri degre show obtain approxim deriv final view pagerank linear oper act prefer vector show tight connect iter comput deriv pictur mongolia extract top element partial order set back long marvel journei thousand pictur friend rel stand dozen choos pain process decid silent vast desert idyl pictur tranquil majest lake pictur pictur pictur summer paolo sebastiano friend part long journei mongolia local guid driver uaz van particip fond take pictur mongolia met match long river flow luxuri meadow high peak cover pine tree wild hors run free children laugh passag camel sandi desert trip back itali moscow short stop start put pictur take week discov hundr digit camera light laptop essenti limit number pictur number scari importantli friend happen invit holidai dinner friend end dinner doubt friend propos pictur take holidai guatemala pictur turn hundr special tree grow jungl choos repres set object easi task natur prefer question larg debat psychometrician economist cf argu point person prefer transit relat experi prove prefer transit commonli agre intransit prefer lead preliminari version paper appear bcv queri dflow graph model applic queri log record queri action user search engin valuabl inform interest prefer behavior user implicit feedback search engin result mine wealth inform queri log import applic includ querylog analysi user profil person advertis queri recommend paper introduc queri flow graph graph represent interest knowledg latent queri behavior intuit queri flow graph direct edg queri qi queri qj mean queri part search mission path queri flow graph search behavior likelihood strength edg path queri flow graph outcom queri log mine time tool propos methodolog build graph mine time textual inform aggreg queri user approach build real world queryflow graph larg scale queri log demonstr util concret applic find logic session queri recommend us queri flow graph applic tempor evolut uk web number convex permutomino number convex permutomino english summari larg time dawar web graph describ techniqu develop gather distribut highli compress access form seri twelv snapshot uk web domain ad hoc compress techniqu made store twelv snapshot bit link constant time access tempor inform collect make studi tempor evolut link base score pagerank growth onlin commun gener time depend phenomena relat link structur pictur mongolia partial sort partial world deeper investig pagerank function damp factor pagerank defin stationari state markov chain chain obtain perturb transit matrix induc web graph damp factor spread uniformli part rank choic emin empir case origin suggest brin page paper give mathemat analysi pagerank show contrarili popular belief real world graph valu close give meaning rank give close form formula pagerank deriv order prove iter power method pagerank obtain maclaurin polynomi degre show obtain approxim deriv final view pagerank linear oper act prefer vector show tight connect iter comput deriv effici optim lazi algorithm minim dinterv semant gener damp function propag import link dbase rank gener damp function propag import link base rank english summari internet math summari paper introduc famili link base rank algorithm propag page import link algorithm includ damp function decreas distanc direct link impli greater endors link longer path pagerank wide rank function famili main object paper determin famili rank techniqu interest se choic damp function affect rank qualiti converg speed result suggest pagerank approxim simpl form rank comput effici focu specul natur aim separ kernel pagerank link base import propag propag decai path focu damp function linear exponenti hyperbol decai length path exponenti decai correspond pagerank gener pagerank damp function link dbase rank algorithm paper introduc famili link base rank algorithm propag page import link algorithm damp function decreas distanc direct link impli endors link long path pagerank wide rank function famili main object paper determin famili rank techniqu interest se choic damp function impact rank qualiti converg speed result suggest page rank approxim simpler form rank comput effici focu specul natur aim separ kernel pagerank link base import propag propag decai path focu damp function linear exponenti hyperbol decai length path exponenti decai correspond pagerank function present includ algorithm analysi comparison experi studi behavior paramet real web graph data result show calcul linear approxim induc page order ident page rank fix small number iter comparison perform kendal larg domain dataset effici lazi algorithm minim dinterv semant mg trec mg trec graph fibrat graph isomorph pagerank pagerank rank method assign score web page limit distribut random walk web graph fibrat graph morphism local isomorph neighbourhood cover project local isomorph neighbourhood show deep connect relat fibrat markov chain restart kind markov chain includ pagerank special case fact constraint valu pagerank assum result show recent defin class graph admit polynomi time isomorph algorithm base comput pagerank subclass fibrat prime graph possess simpl discret polynomi time isomorph algorithm base classic techniqu graph isomorph discuss effici issu implement algorithm case web graph space occup number node accept number arc trap pitfal topic dbias pagerank discuss number issu definit comput comparison pagerank valu address spars literatur contradictori approach studi differ weakli strongli preferenti pagerank patch dangl node distribut extend analyt formula strongli preferenti case corrobor result experi snapshot million page uk domain experi show pagerank version poorli correl result blindli appli comput highlight concern usag exchang base correl indic kendal approxim rank graph fibrat graph isomorph pagerank pagerank rank method assign score web page limit distribut random walk web graph fibrat graph morphism local isomorph neighbourhood cover project local isomorph neighbourhood show deep connect relat fibrat markov chain restart kind markov chain includ pagerank special case fact constraint valu pagerank assum result show recent defin class graph admit polynomi time isomorph algorithm base comput pagerank subclass fibrat prime graph possess simpl discret polynomi time isomorph algorithm base classic techniqu graph isomorph discuss effici issu implement algorithm case web graph space occup number node accept number arc refer collect web spam describ webspam uk collect larg set web page manual annot label indic host includ web spam aspect publicli web spam collect includ page content link label larg divers set judg compress perfect embed skip list quick invert dindex lookup larg invert indic common construct web scale search engin faster access invert indic index intern skip quickli unnecessari document classic approach skip dictat skip posit document pointer number document term appear argu due grow size web refin techniqu describ emb compress perfect skip list invert list provid statist model explain empir distribut skip data observ experi devis good compress techniqu limit wast space result data structur increas index size percent make index pointer fine granular mg trec mg particip track trec ad hoc task effici task terabyt track find relev document high precis million page gov domain time mg group particip trec concentr effort ad hoc task combin techniqu base multi index minim interv semant pagerank pagerank function damp factor pagerank defin stationari state markov chain chain obtain perturb transit matrix induc web graph damp factor spread uniformli part rank choic emin empir case origin suggest brin page recent behaviour pagerank respect discov link spam detect analyt justif chosen miss paper give mathemat analysi pagerank show contrarili popular belief real world graph valu close give meaning rank give close form formula pagerank deriv order extens power method approxim converg deriv final show tight connect iter comput analyt behaviour prove iter power method pagerank obtain maclaurin polynomi degre result pave applic analyt method studi pagerank totalrank rank damp pagerank defin stationari state markov chain obtain perturb transit matrix web graph damp factor spread part rank choic emin empir applic nonetheless select critic link farm choic adversari recent result prove pagerank page ration function function approxim effici fact defin form rank totalrank averag pagerank show rank comput effici provid preliminari experiment result qualiti comparison pagerank languag observatori project lop part paper descript languag observatori project lop highlight major technic difficulti challeng part respond difficulti adopt ubicrawl data collect engin project interact collabor group produc satisfactori result paradox effect pagerank increment comput decid kind visit strategi accumul high qualiti page quickli debat issu design web crawler paper propos relat previous overlook measur effect crawl strategi graph obtain partial visit sens repres underli web graph comput pagerank concern precis interest determin rapidli comput pagerank visit subgraph yield rank agre comput complet graph rank compar kendal describ number larg scale experi show paradox effect visit gather pagerank quickli highest qualiti tend miscalcul pagerank final perform kind experiment analysi synthet random graph gener web graph model result opposit obtain real web graph code world wide web find kind design structur programm examin wide varieti program program sourc code proprietari unavail analysi world wide web web potenti provid rich sourc program studi freeli code web suffici qualiti quantiti provid window softwar design practic todai preliminari studi sourc code avail web estim url object orient sourc code url execut code binari class file repres enorm resourc program analysi risk inaccuraci conserv project sampl result entir web estim web million file java perl sourc code million file sourc code million file execut code keyword design sourc code analysi world wide web estim code world wide web mutabl string java design webgraph framework ii code world dwide web fundament observ compress web graph made construct link databas order url lexicograph order successor list tend small gap code standard method full text index construct paper propos code famili simpl code target gap give rst mathemat compar analysi code power law distribut small expon common gap web graph webgraph thing thought java trade worst make paradox effect pagerank increment comput abstract decid kind visit accumul high qualiti page quickli debatedissu design web crawler breadth visit work tend discov page high pagerank earli crawl visit order depth inturn wors random visit breadth supersed omnisci visit choos step node highest pagerank frontier paper discuss relat previous overlook measur effect crawl strategi graph obtain partial visit sens repres underli web graph asth comput pagerank concern precis interest determin rapidli comput pagerank visit subgraph yield rel rank agre node havein complet graph rank compar kendal describ number larg scale experi show paradox effect visit gather pagerank quickli highest qualiti tend miscalcul pagerank final weperform kind experiment analysi synthet random graph gener web graph model result opposit obtain real web graph introduct search crawl strategi strategi gather earli page high qualiti analmost research topic collect quickli high qualiti page major design goal crawler issu import note crawl abillion page number uncrawl page exce number crawl page basic measur qualiti pagerank variant step cancompar strategi fast cumul pagerank sum pagerank visit page point grow time comparison perform endof crawl graph comput pagerank worst make paradox effect pagerank increment comput abstract decid kind visit accumul high qualiti page quickli debatedissu design web crawler breadth visit work tend discov page high pagerank earli crawl visit order depth inturn wors random visit breadth supersed omnisci visit choos step node highest pagerank frontier paper discuss relat previous overlook measur effect crawl strategi graph obtain partial visit sens repres underli web graph asth comput pagerank concern precis interest determin rapidli comput pagerank visit subgraph yield rel rank agre node havein complet graph rank compar kendal describ number larg scale experi show paradox effect visit gather pagerank quickli highest qualiti tend miscalcul pagerank final weperform kind experiment analysi synthet random graph gener web graph model result opposit obtain real web graph introduct search crawl strategi strategi gather earli page high qualiti analmost research topic collect quickli high qualiti page major design goal crawler issu import note crawl abillion page number uncrawl page exce number crawl page basic measur qualiti pagerank variant step cancompar strategi fast cumul pagerank sum pagerank visit page point grow time comparison perform endof crawl graph comput pagerank webgraph framework compress techniqu studi web graph dicult due larg size recent propos publish techniqu store web graph memori limit space exploit redund web webgraph framework suit code algorithm tool aim make easi manipul larg web graph paper present compress techniqu webgraph centr referenti intervalis turn dual rethink java string java string class string stringbuff lie extrem spectrum immut refer base mutabl content base motiv data intens text applic propos string class mutablestr embodi approach compact approxim lattic function applic larg dalphabet text search propos simpl randomis data structur store approxim lattic valu function comput function requir constant number step error probabl balanc space usag bloom filter structur suit function bottom domain show method store compact bad charact shift function variant boyer moor text search algorithm result obtain practic implement algorithm larg alphabet unicod collat element small setup time lower bound sens direct regular graph lower bound weak sens direct graph vertic maximum degre weak sens direct colour colour suffici conjectur need contrari show suffici larg graph requir log colour give simpl exampl small graph requir colour verifi mechan holograph tree fibrat graph fibrat graph morphism local isomorph neighbourhood cover project local isomorph neighbourhood paper develop systemat theori graph fibrat emphas result recent found applic theori distribut system keyword graph fibrat graph cover graph factor content introduct definit basic properti graph theoret definit fibrat cover group fibrat automorph properti fibrat finit graph applic univers fibrat cover univ univers dynam synchron dstabil prove exist univers synchron stabil protocol protocol distribut system stabil desir nonreact behaviour long protocol stabil behaviour exist previou propos requir drastic increas asymmetri knowledg work protocol addit knowledg requir symmetri break condit stabil respect dynam topolog prove optim quiescenc time synchron network processor diamet protocol made finit state neglig loss quiescenc time optim protocol case uniqu identifi consequ provid effect proof techniqu show stabil behaviour wide rang model keyword stabil anonym network graph fibrat synchron system univers homogen graph dlike structur domain measur jug studi jug problem gener form set jug fix capac find quantiti measur provid upper lower bound number step measur effect character comput anonym network provid effect recurs character relat comput network processor algorithm start state bound network size activ model consid synchron asynchron interleav delta dapproxim function paper studi notion approxim function framework bss model denot function comput bss machin comparison studi class function sens pointwis uniformli main equival result show notion coincid type comput converg speed recurs bound final studi possibl extend result comput archimedean field lower bound weak sens direct case regular graph graph vertic maximum degre weak sens direct colour colour suffici conjectur need colour suffici nonetheless shown suffici larg graph requir log colour paper recent result asymptot graph enumer show surprisingli bound hold regular graph improv log log log show log log colour degre lower bound weak sens direct graph vertic maximum degre weak sens direct colour colour suffici conjectur need contrari show suffici larg graph requir log colour give simpl exampl small graph requir colour verifi mechan cover preserv sens direct sens direct properti label network arc colour graph assign coher local identifi processor basi rout incom messag prove weak sens direct preserv construct regular cover cover induc voltag assign group voltag assign depend colour construct preserv minim kei word distribut system interconnect network sens direct graph cover introduct sens direct properti global consist colour network reduc complex distribut algorithm polynomi algorithm check colour graph weak sens direct polynomi bound high result obviou membership np find colourin ture closur archimedean field bss machin uniform exact test machin equival modulo paramet type ture machin defin notion closur relat ture machin archimedean field show field admit nontrivi uniformli decid set iff ture close partial order set ture close field prove isomorph ideal complet unsolv degre introduct previou paper author introduc version bss model comput exact test allow essenti bss machin uniform iff halt set comput function chang test equal replac test membership arbitrari ball set uniformli semi decid iff halt set uniform bss machin turn set open strict relat uniform comput comput anonym arbitrari knowledg provid character relat comput arbitrari knowledg network processor algorithm start state assum bound network size activ model consid synchron asynchron interleav introduct question problem solv distribut system processor algorithm start state long stori firstli formul angluin investig problem establish center realiz connect theori graph cover provid work yamashita kameda character problem solvabl topolog constraint investig led classif comput function allow elimin severa complex decid sens direct paper prove decid distribut system repres colour digraph node weak sens direct ac processor show decid sens direct algorithm order decid ac colour graph caylei colour graph equal jump ture closur archimedean field bss machin uniform exact test machin equival modulo paramet type ture machin defin notion closur relat ture machin archimedean field show field admit nontrivi uniformli decid set iff ture close partial order set ture close field prove isomorph ideal complet unsolv degre introduct previou paper author introduc version bss model comput exact test allow essenti bss machin uniform iff halt set comput function chang test equal replac test membership arbitrari ball set uniformli semi decid iff halt set uniform bss machin turn set open strict relat uniform comput delta duniform bss machin uniform bss machin standard bss machin reli exact equal test prove archimedean field set uniformli semidecid iff open semi decid bss machin local time bound prove local time bound nontrivi entail number result bss machin exist decid set interior closur semi decid ad constant final show set semidecid ture machin set semi decid uniform machin coeffici field ture comput number introduct problem decid subset real rise complementari approach tradit ture type recurs theori studi set decid gener ture machin input tape represent real number hand bss model consid comput vector function anonym network character vector function comput class anonym network studi undirect bidirect network partial wireless network processor activ model synchron asynchron identifi rigid condit network comput function keyword anonym network wireless network distribut comput graph fibrat introduct network legitim function network comput distribut precis domain valu function number processor network network network class comput question meaning network anonym processor possess uniqu identifi problem elect problem studi ne comput vector function anonym network character vector function comput class anonym network studi undirect bidirect network partial wireless network processor activ model synchron asynchron identifi rigid condit network comput function keyword anonym network wireless network distribut comput graph fibrat introduct network legitim function network comput distribut precis domain valu function number processor network network network class comput question meaning network anonym processor possess uniqu identifi problem elect problem studi ne dstabil univers algorithm prove exist univers stabil algorithm algorithm stabil distribut system desir behaviour long algorithm stabil behaviour exist previou propos requir drastic increas asymmetri knowledg order work algorithm addit knowledg requir symmetri break condit stabil respect topolog identifi assign processor prove tight quiescenc time synchron network processor diamet algorithm made finit state neglig multipl loss activ asynchron propos algorithm quiescenc time result true wide varieti sharedmemori model includ unidirect wireless uniform network introduct system stabil minim sens direct decis problem caylei graph sens direct properti label possibl anonym network assign coher local identifi processor basi rout incom messag graph minim sens direct sens direct number colour equal maximum outdegre prove outregular digraph minim weak sens direct caylei colour graph gener sens requir connected caylei colour graph possess minim transit sens direct obtain character outregular graph minim weak transit sens direct consequ decid colour graph caylei colour graph reduc decid weak sens direct ac symmetri break anonym network character character case elect leader anonym network processor determinist algorithm show network weak elect algorithm elect imposs processor detect fact distribut introduct problem elect leader anonym network processor precis model direct graph vertic processor arc commun link freeli interchang symmetr digraph undirect graph make assumpt structur network selfloop parallel arc allow processor anonym uniqu identifi synchron asynchron processor activ model model port awar local name outgo incom arc unidirect bidirect link good fibrat construct preserv sens direct maxim chain antichain strongli noetherian semiord complex decid sens direct paper prove decid distribut system repres colour digraph node weak sens direct ac processor show decid sens direct algorithm order decid ac colour graph caylei colour graph 